Real-Time Driver Drowsiness Identification Based on Eye-State Analysis

Abstract

Driving while being drowsy has become one of the major reasons of causing road accidents. Drivers who drive at night or for a long distance without resting are more prone to get involved in an accident. Large amount of fatal injuries and deaths occur because of this reason. Hence, it has become an active area of research. Various systems exist for this purpose which makes use of behavioural patterns. Behavioural patterns considered here are visual behaviours of drive like eye blinking, eye closing, yawning, head bending etc. Most of these methods are time consuming and expensive. Recently, in research and development, machine learning methods have been used to predict a driver\'s conditions. Those conditions can be used as information that will improve road safety. Machine Learning has brought progression in video processing which enables images to be analysed with accuracy.

Introduction

I. INTRODUCTION

The aim of this project is to develop a prototype drowsiness detection system. The focus will be placed on designing a system that will accurately monitor the open or closed state of the driver’s eyes in real-time.

By monitoring the eyes, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident and save the lives of many people. Detection of fatigue involves a sequence of images of a face, and the observation of eye movements and blink patterns.

The analysis of face images is a popular research area with applications such as face recognition, virtual tools, and human identification security systems. In this Python project, we will be using OpenCV for gathering the images from webcam and feed them into a deep learning model which will classify whether the person’s eyes are ‘Open’ or ‘Closed’ and detect fatigue. If the system identifies the driver is drowsy an alarm or a signal will be generated to alert the driver.

II. MAIN OBJECTIVE

In this system, we proposed a method for detecting drowsiness by using convolution neural network model over position of eyes using OpenCV and Dlib to count the blinking of eyes.

III. METHODOLOGY

The entire driver drowsiness architecture can be divided into 4 modules.The working of each module can be understood from the following.

A. Face Detection

This module takes input from the camera and tries to detect a face in the video input. The detection of the face is achieved through the Shape_predictor file, by the landmarks using Dlib’s the Frontal face detector function. The face is detected in a rectangle format and converted to grayscale image and stored in the memory which can be used for training the model.

B. Eye Detection

Since the model works on building a detection system for drowsiness we need to focus on the eyes to detect drowsiness. The eyes are detected through the video input by implementing a shape predictor file which is pre-trained where eyes can be obtained from it. The eyes are detected in red outerlines.

C. Eye Tracking

Due to the real-time nature of the project, we need to track the faces continuously for any form of distraction. Hence the faces are continuously detected during the entire time The input to this module is taken from the previous module. The eyes state is determined through Eye Aspect Ratio(EAR) value and also closed eyes in consecutive frames.

D. Drowsiness Detection

In the previous module the frequency is calculated and if it is less than defined threshold value or the frames are found with closed eyes for a longer period then the driver is alerted for the drowsiness through an alert from the system in the form of alarm.

IV. SYSTEM IMPLEMENTATION

A. Face Landmark Detection Mechanism

As you can see from above, we initialize the face landmark detector by using the pretrained model. The model is based on ensemble regression trees because the model will predict continuous numbers. That model is trained on the iBUG-300 W dataset, where it contains images and their corresponding 68 face landmark points. In general, those landmark points belong to the nose, the eyes, the mouth, and the edge of a face.

1. Face Detection: Face detection is the first methods which locate a human face and return a value in x,y,w,h which is a rectangle.
2. Face Landmark: After getting the location of a face in an image, then we have to through points inside of that rectangle.

In our project we need only eyes so the points that are required to eyes are obtained as follows:

After extracting the eye coordinates the EAR(Eye Aspect Ratio) will be calculated and the drowsiness is identified.

B. Eye Aspect Ratio(EAR)

Eye Aspect Ratio Eye Aspect Ratio is proposed by the Czech Technical University. It is a scalar quantity obtained by detecting a face from an image, finding the Euclidean distance of the corresponding eye coordinates, and substituting it into the following formula.Since each eye is represented using 6 landmarks points. The EAR for a single eye is calculated using this formula:

The more the EAR, the more widely eye is open. We would decide a minimum EAR value and used this to decide if the eye is closed or not.

C. Working Procedure

1. Step 1: Initially the OpenCV and Dlib are used for capturing video identifying face of the person and extracting essential features for this a pre trained model shape predictor.dat file is used.
2. Step 2: After identification of face the eyes are detected by using the pre trained model and the eyes are highlighted with red outerline.
3. Step 3: The eyes are identified from the video and then eyes are constantly examined in each frame for detecting drowsiness in consecutive frames.
4. Step 4: If in the consecutive frames the eyes are closed or the Eye Aspect Ratio(EAR) is less than threshold a alarm will be generated.

Our project can give efficient results under normal light conditions and also persons with glasses but the performance may be sometimes varied in poor light conditions.

Thus the system test is done with Python to detect drowsiness of a driver under different conditions.

V. RESULTS

Following are results of drowsiness detection system:

1. The eyes of the driver are detected from the live webcamera and they are highlighted with red outerline.

2. The person is Detected drowsy from live video

VI. FUTURE SCOPE

In the real time driver fatigue detection system we will extend our system to slow down a vehicle automatically when fatigue level crosses a certain limit. Instead of threshold drowsiness level it is suggested to design a continuous scale driver drowsiness detection system. It monitors the level of drowsiness continuously and when this level exceeds a certain value a signal is generated which controls the hydraulic braking system of the vehicle. Since it brings the vehicle speed down to a controllable limit, the chances of accident occurrence is greatly reduced which is quite helpful for avoiding crashes caused by drowsiness related cases.

Conclusion

Driver Drowsiness Detection was built to help a driver stay awake while driving in order to reduce car accidents caused by drowsiness. In order to detect a driver’s drowsiness, facial features, eyes and mouth were identified on the video of an individual driving. Convolutional neural network was implemented to classify eyes as open or closed. Drowsiness was determined on the basis of frequency of closed eyes. Using OpenCV and Dlib in Python, eyes were examined. An alarm was set to ring after the drowsiness detection to alert the driver. The proposed system is observed to work efficiently with person wearing glasses and also under different lighting conditions.

References

[1] https://www.ijert.org/a-machine-learning-approach-for-driver-drowsiness-detection-based-on-eye-state. [2] https://ieeexplore.ieee.org/document/8583852 [3] https://docs.opencv.org/3.4/d7/d8b/tutorial_py_face_detection.html [4] T. Danisman, I. M. Bilasco, C. Djeraba, and N. Ihaddadene, \"Drowsy driver detection system using eye blink patterns,\" in Machine and Web Intelligence (ICMWI), 2010 International Conference on, 2010, pp. 230-233. [5] Python numpy: https://numpy.org/ [6] The Python Standard Library https://python.readthedocs.io/en/latest/library/index. html [7] The MATLAB website. [Online]. Available: http://www.mathworks.com/matlabcentral/fileexchan ge/2 8757-tracking-red-color-objects-using-matlab [8] Pycharm https://www.jetbrains.com/pycharm

Copyright

Copyright © 2022 Navya Chowdary Rayineedu, Afrin Bano, Mounika Uppalapati, Bhargavi Vemula, Chandra Mohan Chetla. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.